Image processing device, image processing method, image processing program and recording medium

ABSTRACT

Occurrence of mosquito noise of a macro block that contains even a small plain region is suppressed. One macro block is subdivided into sub blocks each of which is composed of four pixels×four lines. The average value of luminance levels of each sub block is calculated. The absolute values of the difference values between the luminance levels and the average value are obtained for each sub block. The minimum value of the average difference values of the 16 sub blocks of one macro block is selected and decided as an activity of the macro block. Corresponding to the decided activity, a quantizer scale of the macro block is decided. With the decided quantizer scale, the macro block is quantized. According to this method, since the size of each sub block is smaller than the size of a conventional sub block of eight pixels×eight lines, the average difference value of each sub block is more emphasized. Since the minimum value of the average difference values is designated as an activity of the macro block, an activity of a macro block that contains even a small plain portion can be more suppressed. As a result, a quantizer value of a macro block that contains an edge can be decreased. Thus, mosquito noise of a macro block that contains an edge portion of a picture can be suppressed.

TECHNICAL FIELD

The present invention relates to a picture processing apparatus, apicture processing method, a picture processing program, and a recordingmedium for compression-encoding picture data by quantizing picture datain the unit of one block.

BACKGROUND ART

In a conventional picture data compression-encoding system, picture datais quantized in the unit of one block that is composed of apredetermined number of pixels. For example, in the MPEG2 (MovingPictures Experts Group 2), such a compression-encoding system is used.In the MPEG2, picture data is transformed in the unit of one block thatis composed of a plurality of pixels by the DCT (Discrete CosineTransform) method. The obtained DCT coefficients are quantized. As aresult, the picture data has been compression-encoded. In the MPEG2, aquantizer step is designated with a quantizer scale.

In a conventional picture quality optimizing method using thecompression-encoding of the MPEG2, an activity that is a coefficientthat represents the complexity and smoothness of a picture to becompressed is calculated. With an adaptive quantization based on theactivity, the picture quality is optimized.

This method is performed as follows. In a simple and smooth pictureregion, when the compressing process is performed, the deterioration ofthe picture quality is remarkable. In this region (referred to as planeregion), the picture data is finely quantized with a quantizer scalewhose quantizer step is small. In contrast, in a complicated pictureregion, when the compressing process is performed, the deterioration ofthe picture quality is not remarkable. In this region, the picture datais coarsely quantized with a quantizer scale whose quantizer step islarge. Thus, with a limited code amount, the picture quality can beeffectively optimized.

When picture data is compressed, as described above, each picture regionis divided into pixel blocks each having a predetermined size. Thepicture data is quantized and DCT-transformed in the unit of one pixelblock. The MPEG2 standard prescribes a block of eight pixels×eight linesas the minimum process unit. In addition, the MPEG2 standard prescribesthat each block of eight pixels×eight lines should be DCT-transformedand that the resultant DCT coefficients are quantized in the unit of amacro block of 16 pixels×16 lines.

On the other hand, although the MPEG2 standard does not clearlyprescribe the unit for calculating the forgoing activity, however, theMPEG2 TM5 (Test Model 5) has proposed that the activity should beprocessed in the unit of one sub block of eight pixels×eight lines,which is the same as one DCT block.

Next, the activity calculating method in “adaptive quantizationconsidering visual characteristic” that has been proposed in the MPEG2TM5 will be described.

The adaptive quantization is to vary a quantizer scale Qj that dependson the state of a picture with an activity of each macro block so as tocontrol the generated code amount of for example one frame and improvethe picture quality. The quantizer scale Qj is varied with the activityso that in a plane region, where the deterioration of the picturequality is remarkable, the picture data is quantized with the quantizerscale Qj whose quantizer step is small and in a complicated pictureregion, where the deterioration of the picture quality is notremarkable, the picture data is quantized with the quantizer scale Qjwhose quantizer step is large.

The activity is obtained with pixel values of a luminance signal of anoriginal picture rather than a predictively error. In other words, theactivity is obtained by calculating the following Formulas (1) to (3) inthe reverse order for the j-th macro block with four blocks of the frameDCT encoding mode and four blocks of the field DCT encoding mode.act _(j)=1+min [sblk=1, 8](var _(—) sblk)  (1)var _(—) sblk=1/64Σ[k=1, 64](Pk−Pavg)²  (2)Pavg=1/64Σ[k=1, 64]P _(k)  (3)

where P_(k) is a pixel value of a block of a luminance signal of theoriginal picture. In Formula (3), 64 pixel values of a block of 8×8 aresummed and the result is divided by 64. As a result, the average valuePavg of the pixel values is obtained. Next, in Formula (2), thedifference between the average value Pavg and Pk is obtained. As aresult, the average difference value var_sblk of the block of 8×8 iscalculated. In Formula (1), with the minimum value of the averagedifference values var_sblk, the activity act_(j) of the macro block isobtained. The minimum value is used because even if a part of the macroblock contains a plain portion, it is necessary to finely quantize themacro block.

As described above, in the MPEG2 MT5, an activity is calculated for eachsub block composed of eight pixels×eight lines, which is the same as oneDCT block. The minimum activity of the activities of eight sub block isselected. The selected minimum activity is determined as the activity ofthe macro block.

This method is effective when the compression rate is high and the codeamount is not affordable. However, when the compression rate is low andthe code amount is relatively affordable, if this method is simplyapplied, since the picture quality is high, so-called mosquito noisebecomes remarkable when a macro block contains even a small plainregion.

Now, the mosquito noise will be described in brief. When high frequencycomponents of DCT coefficients are coarsely quantized, at an edgeportion 10A of an object 10 on a screen shown in FIG. 1A, ringing 11shown in FIG. 1B takes place. When the edge portion 10A of the object 10moves, since the separating method for a DCT block 12 varies. As aresult, the state of the ringing 11 varies, which looks like a mosquitoflies around the edge portion 10A. Thus, this phenomenon is calledmosquito noise. When the periphery of the edge portion 10A is a planeregion, the mosquito noise takes place very much.

As described above, when the entire picture is in high quality, ifmosquito noise takes place in a plane region, the mosquito noise becomesmore remarkable. Thus, when the compression rate is low-namely the codeamount is affordable, it is very effective to cause a quantizer scale Qjof a macro block that contains even a small plain region to have a finerquantizer step.

Therefore, an object of the present invention is to provide a pictureprocessing apparatus, a picture processing method, a picture processingprogram, and a recording medium that suppress the occurrence of mosquitonoise in a macro block that contains even a small plain region.

DISCLOSURE OF INVENTION

The present invention is a picture processing apparatus, comprisingtransforming means for performing a transforming process for picturedata in the unit of a transforming block and outputting transformedcoefficients as a result of the transforming process, activity detectingmeans for calculating an activity in the unit of a plurality of subblocks contained in the transforming block of the picture data andoutputting an activity of a block composed of the plurality oftransforming blocks with the activity of each of the sub blocks, andquantizing means for deciding a quantizer scale of the block accordingto the activity output from the activity detecting means and quantizingthe transformed coefficients output from the transforming means with thequantizer scale.

The present invention is a picture processing method, comprising thesteps of performing a transforming process for picture data in the unitof a transforming block and outputting transformed coefficients as aresult of the transforming process, calculating an activity in the unitof a plurality of sub blocks contained in the transforming block of thepicture data and outputting an activity of a block composed of theplurality of transforming blocks with the activity of each of the subblocks, and deciding a quantizer scale of the block according to theactivity output at the activity detecting step and quantizing thetransformed coefficients output at the transforming step with thequantizer scale.

The present invention is a picture processing program for causing acomputer apparatus to execute a picture processing method for quantizinginput picture data in the unit of one block, the picture processingmethod comprising the steps of calculating an activity in the unit of aplurality of sub blocks contained in the transforming block of thepicture data and outputting an activity of a block composed of theplurality of transforming blocks with the activity of each of the subblocks, and deciding a quantizer scale of the block according to theactivity output at the activity detecting step and quantizing thetransformed coefficients output at the transforming step with thequantizer scale.

The present invention is a recording medium on which a pictureprocessing program for causing a computer apparatus to execute a pictureprocessing method for quantizing input picture data in the unit of oneblock has been recorded, the picture processing method comprising thesteps of calculating an activity in the unit of a plurality of subblocks contained in the transforming block of the picture data andoutputting an activity of a block composed of the plurality oftransforming blocks with the activity of each of the sub blocks, anddeciding a quantizer scale of the block according to the activity outputat the activity detecting step and quantizing the transformedcoefficients output at the transforming step with the quantizer scale.

As described, a transforming process is performed for picture data inthe unit of a transforming block. Transformed coefficients as a resultof the transforming process are output. An activity is calculated in theunit of a plurality of sub blocks contained in the transforming block ofthe picture data. An activity of a block composed of the plurality oftransforming blocks with the activity of each of the sub blocks isoutput. A quantizer scale of the block is decided according to theactivity of the block composed of the plurality of transforming blocks.The transformed coefficients are quantized with the quantizer scale.Thus, an activity of each sub block is more emphasized. The quality of apicture that has been adaptively quantized is optimized.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A and FIG. 1B are schematic diagrams for explaining mosquitonoise;

FIG. 2A and FIG. 2B are schematic diagrams for explaining the detectionof activity according to the present invention;

FIG. 3A and FIG. 3B are schematic diagrams for explaining the detectionof an activity corresponding to the MPEG2 TM5;

FIG. 4 is a sample picture for explaining a calculated result of anactivity of a real picture;

FIG. 5 is a schematic diagram showing an example of the distribution ofactivities on a screen corresponding to the MPEG2 TM5;

FIG. 6 is a schematic diagram showing an example of the distribution ofactivities on a screen according to the present invention;

FIG. 7A and FIG. 7B are block diagrams showing the structure of anexample of a digital VTR according to an embodiment of the presentinvention;

FIG. 8A, FIG. 8B, and FIG. 8C are schematic diagrams more practicallyshowing an example of the structure of an MPEG encoder;

FIG. 9A, FIG. 9B, and FIG. 9C are schematic diagrams showing examples ofstructures of streams transferred in each portion of the MPEG encoder;

FIG. 10A, FIG. 10B, and FIG. 10C are schematic diagrams showing examplesof the structures of streams transferred in each portion of the MPEGencoder;

FIG. 11A and FIG. 11B are schematic diagrams showing examples of thestructures of streams transferred in each portion of the MPEG encoder;

FIG. 12A and FIG. 12B are schematic diagrams showing examples of thestructures of streams transferred in each portion of the MPEG encoder;

FIG. 13A, FIG. 13B, and FIG. 13C are schematic diagrams showing examplesof the structures of streams transferred in each portion of the MPEGencoder;

FIG. 14A, FIG. 14B, and FIG. 14C are schematic diagrams showing examplesof the structures of streams transferred in each portion of the MPEGencoder;

FIG. 15 is a schematic diagram showing an example of the structure of astream transferred in each portion of the MPEG encoder;

FIGS. 16A and 16B are schematic diagrams showing an example of a macroblock subdividing process of a sub block segmenting portion;

FIG. 17A, FIG. 17B, and FIG. 17C are schematic diagrams showing anexample of a macro block subdividing process of the sub block segmentingportion after the DCT mode has been determined;

FIG. 18 is a schematic diagram showing an example of the structure ofthe sub block segmenting portion and the activity portion; and

FIG. 19 is a flow chart showing an example of the process of the MPEGencoder is implemented by software.

BEST MODES FOR CARRYING OUT THE INVENTION

Next, an embodiment of the present invention will be described.According to the present invention, the DCT (Discrete Cosine Transform)is performed in the unit of one block of eight pixels×eight lines, ofwhich one macro block of 16 pixels×16 lines as an encoding unit isdivided by four. When the obtained DCT coefficients are quantizedcorresponding to a quantizer scale, the macro block is divided into subblocks each of which is composed of for example four pixels×four lines.An activity of each sub block is detected. Corresponding to the detectedactivities, an activity of each macro block is determined. Correspondingto the activity, a quantizer scale of each macro block is determined.With the determined quantizer scales, macro blocks are adaptivelyquantized.

With reference to FIG. 2A, FIG. 2B, FIG. 3A, and FIG. 3B, the activitydetecting method according to the present invention will be described incomparison with that of the MPEG2 TM5 as the related art. In FIG. 2A, amacro block of 16 pixels×16 lines shows character “3”. In this example,the pixel value of a black pixel is “16”, whereas the pixel value of awhite pixel is “0”. The pixel value represents the luminance level ofthe pixel.

According to the present invention, the activity of a macro block isdetected corresponding to the following procedure. First of all, a macroblock of 16 pixels×16 lines is divided into sub blocks each of which iscomposed of four pixels×four lines. Using Formula (4), an average valueP of luminance levels Yk of each sub block is obtained.P= 1/16Σ_([k=1, 16]) Y _(k)  (4)

Thereafter, using Formula (5), the absolute values of the differencevalues between the luminance levels Yk of pixels of each sub block offour pixels×four lines and the average value P are obtained. Inaddition, the average value of the absolute values of the differencevalues is obtained as an average difference value var_sblk.var _(—) sblk= 1/16Σ_([k=1, 16]) abs(Y _(k) −P)  (5)

One macro block is composed of 16 sub blocks each of which is composedof four pixels×four lines. The minimum value var_sblk is selected fromthe average difference values var_sblk of the 16 sub blocks, whichcompose one macro block. The selected minimum average difference valuevar_sblk is determined as an activity act of the macro block (Formula(6)).act=min_([blk=1, 16])(var _(—) sblk)  (6)

More practically, as shown in FIG. 2A, one macro block of 16 pixels×16lines is divided into sub blocks each of which is composed of fourpixels×four lines. Using Formula (4), the average value P of theluminance levels Yk of each sub block is obtained. For example, a subblock at the upper left corner shown in FIG. 2A has two pixels whoseluminance levels are 16 each and 14 pixels whose luminance levels are 0each. Thus, the average value P=(16×2+0×14)/16=2.

Next, using Formula (5), the absolute values of the differences betweenthe average value P obtained using Formula (4) and the luminance levelvalues of the pixels of the sub block are obtained. As a result, thedifference average value var_sblk is calculated. In the example of thesub block at the upper left corner shown in FIG. 2A, the differenceaverage value var_sblk={abs(16−2)×+abs(0−2)×14}/16=3.5.

FIG. 2B shows the calculated results of Formula (4) and Formula (5) forsub blocks of one macro block. In FIG. 2B, the difference average valuevar_sblk is denoted by “var”. The difference average value var_sblk hasbeen obtained for each sub block. From them, using Formula (6), theminimum difference average value var_sblk is selected. In the exampleshown in FIG. 2B, the difference average value var_sblk of the sub blockat the second rightward position and at the third downward position is“0”, which is the minimum value. Thus, the activity act of the macroblock is determined as “0”.

FIG. 3A and FIG. 3B are schematic diagram for explaining an activitydetected according to the MPEG2 TM5. A picture (character “3”) displayedin a macro block of 16 pixels×16 lines is the same as that shown in FIG.2A. In the MPEG2 TM5, as shown in FIG. 3A, an activity is detected forsub blocks each of which is composed of eight pixels×eight lines, intowhich one macro block of 16 pixels×16 lines is divided.

Next, a sub block at the upper left of the macro block shown in FIG. 3Awill be explained. Using Formula (3) described in the Related Artsection, the average value P_(k) of the luminance levels of the pixelsof the sub block of eight pixels×eight lines is obtained. In theexample, since the number of pixels whose luminance levels are 16 iseight and the luminance levels of the other pixels are 0. Thus, theaverage value P_(k)=(16×8+0×56)/64=2. Using Formula (2), the differenceaverage value var_sblk of the sub blocks becomesvar_sblk={abs(16−2)×8+abs(0−2)×56}/64=3.5.

FIG. 3B shows the calculated results of Formula (3) and Formula (2) forsub blocks of one macro block. In FIG. 3B, the difference average valuevar_sblk is denoted by “var”. The minimum difference average valuevar_sblk=3.1 is selected from the difference average values var_sblk ofthe sub blocks. Thus, the activity act of the macro block is determinedas “3.1”.

Thus, when the activity calculating method according to the presentinvention is applied to a macro block that contains even a small plainportion, the activity value of the macro block can be more suppressedthan that of the MPEG2 TM5 as the related art. In other words, accordingto the present invention, since a macro block is divided into sub blockswhich are smaller than those according to the MPEG2 TM5, the probabilitythat a plain portion of a macro block is detected becomes high. Thus,the activity of a macro block that contains a so-called edge portionsuch as a character or a contour portion of an image can be suppressed.As a result, mosquito noise can be suppressed.

Next, with reference to FIG. 4, FIG. 5, and FIG. 6, the calculatedresult of an activity of a real picture will be described. A samplepicture shown in FIG. 4 is entirely plane. In particular, the picturecontains edges at boarder portions of patterns drawn on the backgroundportion and character portions of a calendar.

FIG. 5 and FIG. 6 show the distribution of activities of the samplepicture shown in FIG. 4 according to the MPEG2 TM5 and that according tothe present invention, respectively. In FIG. 5 and FIG. 6, diameters ofblack circles represent activity values of macro blocks of the pictures.

When the activities according to the MPEG2 TM5 shown in FIG. 5 and theactivities according to the present invention shown in FIG. 6 arecompared, it is clear that the activities according to the presentinvention are suppressed to smaller values than those according to theMPEG2 TM5. With reference to FIG. 5, there are macro blocks that havevery large activities in a plain portion (around at X=42 and Y=18 to 23)shown in FIG. 5, which is a center right portion of the calendarportion. In this portion, the quantization is coarsely performed. As aresult, it is supposed that the mosquito noise becomes remarkable. Incontrast, with reference to FIG. 6, the activities of the calendarportion according to the present invention are entirely suppressed. As aresult, since the quantization is finely performed, the mosquito noisebecomes unremarkable.

FIG. 7A and FIG. 7B show an example of the structure of a digital VTRaccording to an embodiment of the present invention. The digital VTR isdesigned to directly record a digital video signal that has beencompression-encoded according to the MPEG system to a recording medium.

First of all, the structure and the processing operation of a recordingsystem of the digital VTR will be described. Signals that are input fromthe outside to the recording system are two types of serial digitalinterface signals that are an SDI (Serial Data Interface) signal and anSDTI (Serial Data Transport Interface) signal, an analog interfacesignal, and an external reference signal REF that is a control signal.

The SDI is an interface prescribed by the SMPTE so as to transmit a(4:2:2) component video signal, a digital audio signal, and additionaldata. The SDTI is an interface through which an MPEG elementary stream(referred to as MPEG ES) that is a stream of which a digital videosignal has been compression-encoded according to the MPEG system. The ESis 4:2:2 components. As described above, the ES is a stream of all Ipictures having the relation of 1 GOP=1 picture. In the SDTI-CP (ContentPackage) format, the MPEG ES is separated into access units and packedto a packet in the unit of one frame. The SDTI-CP uses a sufficienttransmission band (clock rate: 27 MHz or 36 MHz or stream bit rate: 270Mbps or 360 Mbps). In one frame period, the ES can be transmitted as aburst.

The SDI signal transmitted through the SDI is input to an SDI inputportion 101. An analog input signal as an analog video signal is inputto an analog input portion 120. The analog input portion 120 convertsthe analog input signal into a digital signal, maps the digital signalto for example the aforementioned SDI format, and outputs the resultantSDI signal. The SDI signal, where the analog input signal has beenconverted and mapped to the SDI format, is supplied to the SDI inputportion 101.

The SDI input portion 101 converts the supplied SDI signal as a serialsignal into a parallel signal. In addition, the SDI input portion 101extracts an input synchronous signal as an input phase reference fromthe SDI signal and outputs the extracted input synchronous signal to atiming generator TG 102.

In addition, the SDI input portion 101 separates a video signal and anaudio signal from the converted parallel signal. The separated videoinput signal and audio signal are output to an MPEG encoder 103 and adelay circuit 104, respectively.

The timing generator TG 102 extracts a reference synchronous signal froman input external reference signal REF. In synchronization with adesignated one of the reference synchronous signal or the inputsynchronous signal, which has been supplied from the SDI input portion101, the timing generator TG 102 generates a timing signal necessary forthe digital VTR and supplies it as timing pulses to each block.

The MPEG encoder 103 converts the input video signal into coefficientdata according to the DCT method, quantizes the coefficient data, andencodes the quantized data with a variable length code.Variable-length-coded (VLC) data, which has been output from the MPEGencoder 103, is an elementary stream (ES) according to the MPEG2. Theelementary stream, which has been output from the MPEG encoder 103, issupplied to one of input terminals of a recording side multi-formatconverter (hereinafter referred to as recording side MFC).

The delay circuit 104 functions as a delay line that delays the inputaudio signal as non-compressed signal corresponding to the delay of thevideo signal of the MPEG encoder 103. The audio signal, which has beendelayed by the delay circuit 104, is output to an ECC encoder 107. Thisis because the digital VTR according to the embodiment of the presentinvention treats the audio signal as a non-compressed signal.

The SDTI signal supplied from the outside through the SDTI is input toan SDTI input portion 105. The SDTI input portion 105 detects thesynchronization of the SDTI signal. The SDTI signal is temporarilybuffered and the elementary stream is extracted therefrom. The extractedelementary stream is supplied to the other input terminal of therecording side MFC 106. The synchronous signal, which has beensynchronously detected, is supplied to the timing generator TG 102 (notshown).

The SDTI input portion 105 extracts a digital audio signal from theinput SDTI signal. The extracted digital audio signal is supplied to theECC encoder 107.

In the digital VTR according to the present invention, an MPEG ES can bedirectly input independent from the base band video signal, which hasbeen input from the SDI input portion 101.

The recording side MFC 106 has a stream converter and a selector. Therecording side MFC 106 selects one of the MPEG ES supplied from the SDIinput portion 101 and the MPEG ES supplied from the SDTI input portion105, and collects DCT coefficients of DCT blocks of one macro block sothat frequency components are rearranged in the order of frequencycomponents. The resultant stream, of which the coefficients of the MPEGES have been rearranged, is referred to as converted elementary stream.Since the MPEG ES is rearranged, when a search reproduction isperformed, as many DC coefficients and low order AC coefficients can becollected as possible, which contributes the improvement of the qualityof the search picture. The converted elementary stream is supplied tothe ECC encoder 107.

A main memory (not shown) having a large storage capacity is connectedto the ECC encoder 107. The ECC encoder 107 has a packing and shufflingportion, an audio outer code encoder, a video outer code encoder, aninner code encoder, an audio shuffling portion, a video shufflingportion, and so forth. In addition, the ECC encoder 107 has an syncblock ID adding circuit and a synchronous signal adding circuit.According to the first embodiment of the present invention, as an errorcorrection code for the video signal and audio signal, a product code isused. In the product code, a data symbol is dually encoded in such amanner that the two-dimensional array of the video signal or audiosignal is encoded in the vertical direction with an outer code and thatthe two-dimensional array is encoded in the horizontal direction with aninner code. As the outer code and inner code, the Reed-Solomon code canbe used.

The converted elementary stream, which has been output from therecording side MFC 106, is supplied to the ECC encoder 107. In addition,the audio signals, which are output from the SDTI input portion 105 andthe delay circuit 104, are supplied to the ECC encoder 107. The ECCencoder 107 shuffles the converted elementary stream and audio signals,encodes them with an error correction code, adds IDs and a synchronoussignal to sync blocks, and outputs the resultant signal as record data.

The record data, which has been output from the ECC encoder 107, isconverted into a record RF signal by the equalizer EQ 108 that has arecording amplifier. The record RF signal is supplied to a rotating dram109. The rotating dram 109 records the record signal on a magnetic tape110. The rotating dram 109 has a rotating head disposed in apredetermined manner. In reality, a plurality of magnetic heads whoseazimuths are different from each other and that form adjacent tracks aredisposed.

When necessary, the record data may be scrambled. When data is recorded,it may be digitally modulated. In addition, the partial response class 4and Viterbi code may be used. The equalizer EQ 108 has both a recordingside structure and a reproducing side structure.

Next, the structure and the processing operation of the reproducingsystem of the digital VTR will be described. In the reproduction mode, areproduction signal that is reproduced from the magnetic tape 110 by therotating dram 109 is supplied to the reproducing side structure of theequalizer EQ 108 that has a reproducing amplifier and so forth. Theequalizer EQ 108 equalizes the reproduction signal and trims the waveshape thereof. When necessary, the equalizer EQ 108 demodulates thedigital modulation and decodes the Viterbi code. An output of theequalizer EQ 108 is supplied to an ECC decoder 111.

The ECC decoder 111 performs the reverse process of the ECC encoder 107.The ECC decoder 111 has a main memory having a large storage capacity,an inner code decoder, an audio deshuffling portion, a video deshufflingportion, and an outer code decoder. In addition, the ECC decoder 111 hasa deshuffling and depacking portion and a data interpolating portion forvideo data. Likewise, the ECC decoder 111 has an audio AUX separatingportion and a data interpolating portion for audio data.

The ECC decoder 111 detects the synchronization of reproduction data. Inother words, the ECC decoder 111 detects a synchronous signal added atthe beginning of a sync block and extracts the sync block from thereproduction signal. The ECC decoder 111 corrects an error of each syncblock of the reproduction data with an inner code. Thereafter, the ECCdecoder 111 performs an ID interpolating process for each sync block.The ECC decoder 111 separates video data and audio data from thereproduction data, where IDs have been interpolated. The ECC decoder 111deshuffles the video data and audio data so that they are restored tothe original data. The ECC decoder 111 corrects an error of thedeshuffled data with an outer code.

When the ECC decoder 111 cannot correct an error of data, which exceedsits error correcting performance, the ECC decoder 111 sets an error flagto the data. For an error of video data, the ECC decoder 111 outputs asignal ERR that represents that data has an error.

The reproduction audio data, whose error has been corrected, is suppliedto an SDTI output portion 115. A delay circuit 114 delays thereproduction audio data for a predetermined amount and supplies thedelayed reproduction audio data to an SDI output portion 116. The delaycircuit 114 absorbs the delay of the video data processed in an MPEGdecoder 113 that will be described later.

On the other hand, the video data, whose error has been corrected, issupplied as converted reproduction elementary stream to a reproducingside MFC circuit 112. The aforementioned signal ERR is also supplied tothe reproducing side MFC circuit 112. The reproducing side MFC circuit112 performs the reverse process of the recording side MFC 106. Thereproducing side MFC circuit 112 has a stream converter. The streamconverter performs the reverse process of the recording side streamconverter. In other words, the stream converter rearranges DCTcoefficients arranged in the order of frequency components to thosearranged in the order of DCT blocks. As a result, the reproductionsignal is converted into an elementary stream according to the MPEG2. Atthat point, when the signal ERR is supplied from the ECC decoder 111 tothe reproducing side MFC circuit 112, it replaces the corresponding datawith a signal that perfectly complies with the MPEG2.

The MPEG ES, which has been output from the reproducing side MFC circuit112, is supplied to the MPEG decoder 113 and the SDTI output portion115. The MPEG decoder 113 decodes the supplied MPEG ES so as to restoreit to the original video signal, which has not been compressed. In otherwords, the MPEG decoder 113 performs an inversely quantizing process andan inverse DCT process for the supplied MPEG ES. The decoded videosignal is supplied to an SDI output portion 116.

As described above, the audio data, which had been separated from thevideo data by the ECC decoder 111, has been supplied to the SDI outputportion 116 through the delay circuit 114. The SDI output portion 116maps the supplied video data and audio data in the SDI format so as toconvert them into the SDI signal having the SDI data structure. The SDIsignal is output to the outside.

On the other hand, as described above, the audio data, which had beenseparated from the video data by the ECC decoder 111, has been suppliedto the SDTI output portion 115. The SDTI output portion 115 maps thevideo data and audio data as the supplied elementary stream in the SDTIformat so as to convert them into the SDTI signal having the SDTI datastructure. The SDTI signal is output to the outside.

A system controller 117 (abbreviated as sys-con 117 in FIG. 7A and FIG.7B) is composed of for example a micro computer. The system controller117 communicates with each block using a digital signal SY_IO so as tocontrol the entire operation of the digital VTR. A servo 118communicates with the system controller 117 using a signal SY_SV. Usingthe signal SV_IO, the servo 118 controls the traveling of the magnetictape 110 and drives and controls the rotating dram 109.

FIG. 8A, FIG. 8B, and FIG. 8C more practically show the structure of theforgoing example of the MPEG encoder. FIG. 9A, FIG. 9B, FIG. 9C, FIG.10A, FIG. 10B, FIG. 10C, FIG. 11A, FIG. 11B, FIG. 12A, FIG. 12B, FIG.13A, FIG. 13B, FIG. 13C, FIG. 14A, FIG. 14B, FIG. 14C, and FIG. 15 showexamples of the structures of streams transferred in the individualportions of FIG. 8A, FIG. 8B, and FIG. 8C.

The MPEG encoder 103 is composed of an input field activity averagingprocess portion 103A, a pre-encoding process portion 103B, and an encodeportion 103C. The input field activity averaging process portion 103Aobtains the average value of activities of the input video data andsupplies the obtained average value to the pre-encoding process portion103B. The pre-encoding process portion 103B estimates the generated codeamount of quantized input video data with the average value of theactivities. According to the estimated result, while controlling thecode amount, the encode portion 103C actually quantizes the input videodata, encodes the quantized video data with a variable length code, andoutputs the resultant data as an MPEG ES.

A timing generator 220 generates a timing signal necessary for the MPEGencoder 103 with a horizontal synchronous signal HD, a verticalsynchronous signal VD, and a field synchronous signal FLD that have beensupplied from for example the timing generator TG 102 shown in FIG. 7. ACPU I/F block 221 is an interface with the system controller 117 shownin FIG. 7. With a control signal and data transferred through the CPUI/F block 221, the operation of the MPEG encoder 103 is controlled.

First of all, the process of the input field activity averaging processportion 103A will be described. Video data, which has been output fromthe SDI input portion 101 and input to the MPEG encoder 103, is suppliedto an input portion 201. The input field activity averaging processportion 103A converts the video data into data that can be stored in amain memory 203 and checks a parity for the video data. The video data,which has been output from the input portion 201, is supplied to aheader creating portion 202. Using a vertical blanking region or thelike, headers according to the MPEG, for example sequence_header,quantizer_matrix, and gop_header, are extracted. The extracted headersare stored in the main memory 203. These headers are designated mainlyby the CPU I/F block 221. In other than the vertical blanking region, inthe header creating portion 202, the video data supplied from the inputportion 201 is stored in the main memory 203.

The main memory 203 is a frame memory for a picture. The main memory 203rearranges video data and absorbs the system delay. The video data isrearranged by for example an address controller (not shown) thatcontrols read addresses of the main memory 203. In FIG. 8A, FIG. 8B, andFIG. 8C, 8 lines, 0.5 frame, and 1 frame in the block of the main memory203 represent delay values as read timings of the main memory 203. Theyare properly controlled corresponding to a command issued from thetiming generator TG 220.

A raster scan/block scan converting portion 204 extracts macro blocksaccording to the MPEG from each line of the video data, which has beenstored in the main memory 203, and supplies the extracted macro blocksto a sub block forming portion 230 disposed downstream thereof.According to the embodiment, with only the first field, activities arecalculated. Thus, the macro blocks that are output from the rasterscan/block scan converting portion 204 are composed of video data of thefirst field.

As shown in FIG. 9A, at the beginning of the stream that is output fromthe raster scan/block scan converting portion 204, address informationof the vertical and horizontal directions of the macro block are placed.The address information is followed by a blank area having apredetermined size, followed by picture data for one macro block.

The stream has a data length of 576 words each of which is composed offor example eight bits. The last 512 words (referred to as data portion)are assigned a picture data area for one macro block. The first 64 words(referred to as header portion) contain the aforementioned addressinformation of the macro block. The other portion is a blank area fordata and flags embedded by each portion disposed downstream of theraster scan/block scan converting portion 204.

A macro block according to the MPEG is a matrix of 16 pixels×16 lines.However, as described later with reference to FIGS. 16A and 16B, theinput field activity averaging process portion 103A performs the processfor obtaining activities with only the first field. Thus, when eightlines of the first field have been stored in the main memory 203, theprocess can be started. In reality, corresponding to a command issuedfrom the timing generator TG 220, the process is properly started.

The sub block forming portion 230 subdivides the macro block of onefield supplied from the raster scan/block scan converting portion 204into sub blocks. FIG. 16A and FIG. 16B show an example of the macroblock subdividing process of the sub block forming portion 230. As shownin FIG. 16A, the macro block, which has been supplied from the rasterscan/block scan converting portion 204, is composed of four DCT blocks,each of which is composed of eight pixels×eight lines. Since the inputfield activity averaging process portion 103A obtains the averageactivity using only the first field, line data of the second fieldhatched does not exist.

The macro block of 16 pixels×16 lines is subdivided into sub blocks eachof which is composed of four pixels×four lines as shown on the left sideof FIG. 16B. Finally, 16 sub blocks of only data of the first field areformed as shown in FIG. 16B. Each of the 16 sub blocks is composed offour pixels×two lines.

An output of the sub block forming portion 230 is supplied to anactivity portion 205. The activity portion 205 obtains field_act of eachmacro block of the first field. Using Formula (7), the average value Pof the luminance levels Yk of four pixels×two lines of each sub block isobtained.P=⅛Σ[k=1, 8]yk  (7)

Next, using Formula (8), the absolute values of the difference valuesbetween the luminance levels Yk of the pixels and the average value P ofeach sub block are obtained. Next, the average difference value var_sblkof the absolute values of the difference values is obtained.var _(—) sblk=⅛Σ[k=1, 8]abs(Yk−P)  (8)

Since one macro block is composed of 16 sub blocks, the minimum value isselected from 16 average difference values var_sblk of 16 sub blocks(Formula (9)). The selected average difference value var_sblk isreferred to as the activity field_act of the macro block of the firstfield.field_act=min[blk=1, 16](var_sblk)  (9)

The activities field_act of macro blocks of the first field are suppliedto an averaging portion 206. The averaging portion 206 cumulates theactivities field_act of the macro blocks of the first field and obtainsthe average value avg_act of the activities field_act (the average valueis referred to as average activity) (Formula (10)). In Formula (10),MBnum represents the total number of macro blocks in one frame.avg_act=1/MBnumΣ[m=1. MBnum]field_act[m]  (10)

The average value avg_act is supplied to a normalizing portion 233 (thatwill be described later) of the pre-encoding process portion 103B. Aswill be described later, with the average value avg_act, the normalizingportion 233 normalizes the activity of each macro block. The normalizingportion 233 performs the normalizing process with the average valueavg_act for the first field and the second field. Thus, after theaverage value avg_act of the activities of the first field has beenobtained, with the average value, the pre-encoding process can beperformed in consideration of the adaptive quantization.

Next, the pre-encoding process portion 103B will be described. A rasterscan/block scan converting portion 207A performs basically the sameprocess as the raster scan/block scan converting portion 204. However,the raster scan/block scan converting portion 207A requires video dataof both the first field and second field to perform the pre-encodingprocess for estimating the code amount. Thus, when eight lines of thesecond field have been stored to the main memory 203, the rasterscan/block scan converting portion 207A can form a macro block of 16pixels×16 lines according to the MPEG. At that time, the process can bestarted. In reality, corresponding to a command issued from the timinggenerator TG 220, the process is properly started.

Video data that has been output from the raster scan/block scanconverting portion 207A is supplied to a DCT mode portion 208. The DCTmode portion 208 decides to use the field DCT encoding mode or the frameDCT encoding mode to encode video data.

In this example, the sum of absolute values of difference values ofadjacent pixels in the vertical direction is calculated in both thefield DCT encoding mode and the frame DCT encoding mode. The sumcalculated in the field DCT encoding mode is compared with the sumcalculated in the frame DCT encoding mode. The encoding mode with thesmaller value is selected. The selected result is temporarily insertedas DCT mode type data that is a flag in the stream that is sent to eachportion downstream. As shown in FIG. 9B, the DCT mode type data dct_typis placed on the rear end side of the blank area of the header portion.

A macro block that has been output from the DCT mode portion 208 issupplied to a sub block forming portion 231. The sub block formingportion 231 subdivides the supplied macro block into sub blocks each ofwhich is composed of four pixels×four lines.

At that point, the DCT mode type data dct_typ inserted into the suppliedstream is referenced. When it has been determined that the DCT mode typedct_typ represents the frame encoding mode, the macro block shown inFIG. 17A is subdivided into sub blocks each of which is composed of fourpixels×four lines and that alternately contain lines of the first andsecond fields. When it has been determined that the DCT mode type datadct_typ represents the field encoding mode, as shown in FIG. 17C, themacro block is subdivided into sub blocks of the first field and subblocks of the second field, each sub block being composed of fourpixels×four lines.

In FIG. 17A, FIG. 17B, and FIG. 17C, hatched lines represent the secondfield, whereas non-hatched lines represent the first field.

The macro block, which has been subdivided into sub blocks, is suppliedto an activity portion 232. The activity portion 232 obtains the averagevalue P of the luminance levels Yk of each sub block of four pixels×fourlines using Formula (11).P= 1/16Σ[k=1, 16]Yk  (11)

Next, the absolute values of the difference values between the luminancelevels Yk of each sub block and the average value P are obtained. Inaddition, the average difference value var_sblk of the absolute valuesof the difference values of each sub block is obtained.var _(—) sblk= 1/16Σ[k=1, 16]abs(Yk−P)  (12)

Since one macro block is composed of 16 sub blocks, the minimum value of16 average difference values var_sblk of each sub block is selected(Formula (13)). The selected average difference value var_sblk isreferred to as activity act of the first and second fields of the macroblock.act=min [blk=1, 16](var_sblk)  (13)

As shown in FIG. 9C, the activity act of the macro block is placed inthe header portion of the stream. The activity act is supplied to thenormalizing portion 233 along with data of the macro block. With theactivity act of the macro block and the average value avg_act of thefield activities obtained from the averaging portion 206, thenormalizing portion 233 obtains a normalized activity Nact.

The normalized activity Nact can be obtained using Formula (14) with forexample the activity act of the macro block and the average valueavg_act of the field activities so that the normalized activity Nact isin the range from “2.0” to “0.5”.Nact=(2×act+avg_act)/(act+2×avg_act)  (14)

As shown in FIG. 10A, the normalized activity Nact is temporarilyinserted as normalized activity data norm_act that is a flag into theheader portion of the stream and sent to each portion downstream. Theaforementioned activity act is overwritten to the normalized activitydata norm_act in the stream.

An output of the normalizing portion 233 is supplied to a DCT portion210A. The DCT portion 210A divides the supplied macro block into DCTblocks each of which is composed of eight pixels×eight lines, performsthe two-dimensional DCT for each DCT block, and generates DCTcoefficients. As shown in FIG. 10B, the DCT coefficients are placed inthe data portion of the stream and supplied to a quantizer table portion211A.

The quantizer table portion 211A quantizes the DCT coefficients, whichhave been transformed by the DCT portion 210A, with a quantizer matrix(quantizer_matrix). As shown in FIG. 10C, the DCT coefficients, whichhave been quantized by the quantizer table portion 211A, are placed inthe data portion of the stream and then output. An output of thequantizer table portion 211A is supplied to a multi-staged quantizerportion composed of a plurality of Q_n (quantizer) portions 212, 212, .. . , VLC portions 213, 213, . . . , cumulating portions Σ 214, 214, andcumulating portions Σ 215, 215, . . . . The DCT coefficients quantizedby the quantizer table portion 211A are quantized on multiple stages ofthe quantizer portions.

The Q_n portions 212, 212, . . . quantize the DCT coefficients withdifferent quantizer scales (quantizer_scale) Q. The values of thequantizer scales Q are prescribed in for example the MPEG2 standard. TheQ_n portions 212, 212, . . . are composed of for example 31 quantizersaccording to the standard. At that point, since n=31, the Q_n portions212, 212, . . . are a Q_1 portion, a Q_2 portion, . . . , and a Q_31portion. With the quantizer scales Qn, the Q_n portions 212 quantize DCTcoefficients with the quantizer scales Qn assigned thereto at a total of31 steps. Hereinafter, the quantizer scale values of the Q_n portions212, 212, . . . are denoted by the quantizer scales Qn values.

The Q_n portions 212, 212, . . . quantize DCT coefficients with theirquantizer scale Qn values. At that point, with the quantizer scalemqaunt, which is in consideration of the visual characteristic, andwhich has been obtained using Formula (6) with the normalized activitydata norm_act obtained by the normalizing portion 233, the adaptivequantization is performed.mqaunt=Q _(—) n×norm_act  (6)

The DCT coefficients adaptively quantized by the Q_n portions 212, 212,. . . with the quantizer scales Qn are placed in the data portion of thestream as shown in FIG. 11A and supplied to the VLC portions 213, 213, .. . . The VLC portions 213, 213, . . . scan DCT coefficients for theindividual quantizer scales Qn according to for example the zigzagscanning method and encodes them with a variable length code withreference to a VLC table according to for example the two-dimensionalHuffman code.

The data, which has been encoded with the variable length code by theVLC portions 213, 213, . . . , is placed in the data portion of thestream as shown in FIG. 11B and then output. Outputs of the VLC portions213, 213, . . . are supplied to the corresponding cumulating portions Σ214, 214, . . .

The cumulating portions Σ 214, 214, . . . cumulate the generated codeamounts for each macro block. As described above, when 31 types ofquantizing devices are used, 31 types of generated code amounts areobtained for each macro block. As shown in FIG. 12A, the generated codeamounts cumulated by the cumulating portions Σ 214, 214, . . . areplaced in the header portion of the stream. In other words, thegenerated code amounts quantized by the Q_1 portion 212 to Q_n portion212 for each macro block are placed in the header portion of the stream.The data portion of the stream is deleted. The stream of each macroblock is supplied to the main memory 203.

The generated code amounts for each macro block, which have been outputfrom the cumulating portions Σ 214, 214, . . . , are supplied to therespective cumulating portions Σ 215, 215, . . . . The cumulatingportions Σ 215, 215, . . . select generated code amounts for each macroblock quantized with quantizer_scale (=mquant), in which the forgoingvisual characteristic has been considered, from those obtained by thecumulating portions Σ 214, 214, . . . and cumulate them for one frame.

The generated code amounts (frame data rates) cumulated by thecumulating portions Σ 215, 215, . . . for the quantizer scales Qn aresupplied as an n-word stream as shown in FIG. 12B to a rate controllingportion 217. When 31 types of quantizing devices are used as theforgoing example, corresponding 31 types of generated code amounts foreach frame are obtained.

Next, a method for obtaining a generated code amount will be describedmore practically. For example, “generated code amount of Q_4 portion212” can be obtained as follows.

For example, in the case

norm_act [1]=1.3

norm_act [2]=1.5

norm_act [3]=0.8

norm_act [4]=1.0

. . .

mqaunt [1]=4×1.3=5.2

: The generated code amount of the Q_5 portion 212 is obtained from theheader portion of FIG. 12A.

mqaunt [2]=4×1.5=6.0

: The generated code amount of the Q_6 portion 212 is obtained from theheader portion of FIG. 12A.

mqaunt [3]=4×0.8=3.2

: The generated code amount of the Q_3 portion 212 is obtained from theheader portion of FIG. 12A.

mqaunt [4]=4×1.0=4.0

: The generated code amount of the Q_4 portion 212 is obtained from theheader portion of FIG. 12A.

. . .

They are cumulated for one frame. They are performed for each of the Q_1portion 212 to the Q_n portion 212. As a result, the generated codeamount for one frame is obtained.

Next, the encoding process portion 103C will be described. The encodingprocess portion 103C performs the final encoding process. As describedabove, the pre-encoding process portion 103B estimates the generatedcode amounts for one frame in various quantizing operations. Theencoding process portion 103C encodes data corresponding to thegenerated code amount, which has been estimated for one frame so thatthe generated code amount does not exceed the pre-designated targetgenerated code amount and outputs an MPEG ES.

The data used in the encoding process portion 103C has been stored inthe main memory 203. However, as described above, when the generatedcode amounts for one frame have been estimated in various quantizingoperations by the pre-encoding process portion 103B, the encodingprocess portion 103C can start the process. As described above, theprocess of each portion of the encoding process portion 103C can beproperly started corresponding to a command issued from the timinggenerator TG 220.

Video data that has been read from the main memory 203 is supplied to araster scan/block scan converting portion 207B. The raster scan/blockscan converting portion 207B performs a process similar to the processof the raster scan/block scan converting portion 207A and extracts amacro block of 16 pixels×16 lines from the video data. As shown in FIG.13A, the extracted macro block is placed in a data portion correspondingto the header portion shown in FIG. 12A and supplied to a DCT modeportion 216.

Like the DCT mode portion 208, the DCT mode portion 216 decides to usethe field DCT encoding mode or the frame DCT encoding mode to encodedata. At that point, the DCT mode portion 208 has decided the DCT modeportion 208 and temporarily placed the result as DCT type data dct_typin the stream (see FIG. 13A). The DCT mode portion 216 detects the DCTtype data dct_typ from the stream and switches to the field encodingmode or the frame encoding mode corresponding to the detected DCT typedata dct_typ. An output of the DCT mode portion 216 is shown in FIG.13B.

A macro block that has been output from the DCT mode portion 216 issupplied to a DCT portion 210B. Like the DCT portion 210A, the DCTportion 210B two-dimensionally transforms the macro block into DCTcoefficients in the unit of one DCT block of eight pixels×eight lines.As shown in FIG. 13 c, the DCT coefficients, into which the macro blockhas been two-dimensionally transformed corresponding to thetwo-dimensional DCT method, are placed in the data portion of the streamand then output from the DCT portion 210B.

A quantizer table portion 211B can be structured in the same manner asthe forgoing quantizer table portion 211A. The quantizer table portion211B quantizes the DCT coefficients transformed by the DCT portion 210Bwith a quantizer matrix. As shown in FIG. 14A, the DCT coefficientsquantized by the quantizer table portion 211B are placed in the dataportion of the stream and supplied to a rate controlling portion 217.

The rate controlling portion 217 selects one from the frame data rates,which have been obtained by the cumulating portions Σ 215, 215, . . . ofthe pre-encoding process portion 103B for each quantizer scale Qn sothat the selected one does not exceed the maximum generated code amountper frame designated by the system controller 117 and is the closest tothe designated value. The quantizer scale (mquant) for each macro blockused in the quantizing device corresponding to the selected frame datarate is obtained from the normalized activity data norm_act placed inthe stream and supplied to a quantizing portion 218.

The quantizer scale for each macro block is placed as quantizer_scale onthe rear end side of the header portion of the stream as shown in FIG.14B and then sent to the quantizing portion 218.

The maximum generated code amount per frame is designated by for examplethe system controller 117 and supplied to the rate controlling portion217 through the CPU I/F block 221.

At that point, the value of the quantizer scale (mquant) for each macroblock can be decreased by one size in the range that does not exceed thedifference between the maximum generated code amount per frame, whichhas been designated by the system controller 117 and transferred throughthe CPU I/F block 221, and the generated code amount corresponding tothe quantizer scale (mquant) for each macro block, which has beenobtained from the normalized activity data norm_act placed in thestream. Thus, since a code amount close to the maximum generated codeamount designated by the system controller 117 and transferred troughthe CPU I/F block 221 is obtained, high picture quality can beaccomplished.

The quantizing portion 218 extracts the quantizer scale(quantizer_scale) designated by the rate controlling portion 217 in theforgoing manner from the stream and quantizes DCT coefficients with thequantizer table portion 211B corresponding to the extracted quantizerscale. At that point, since the quantizer scale supplied from the ratecontrolling portion 217 is the value of the quantizer scale (mquant)obtained from the normalized activity data norm_act, adaptivequantization is performed in consideration of visual characteristic.

The DCT coefficients quantized by the quantizing portion 218 are placedin the data portion of the stream as shown in FIG. 14C and supplied to aVLC portion 219. The DCT coefficients supplied to the VLC portion 219are scanned corresponding to for example the zigzag scanning method. Theresultant DCT coefficients are encoded with a variable length code withreference to a VLC table according to the two-dimensional Huffman code.The variable length code is bit-shifted so that it is byte-aligned andthen output as an MPEG ES.

At that point, as shown in FIG. 15, the header portion as the first halfportion of the stream is replaced with the MPEG header portion in whichthe MPEG header information of the slice layer or below is placed. Thevariable length code is placed in the data portion on the second halfside of the stream.

FIG. 18 shows the structure of an example of the above-described subblock forming portion 231 and the activity portion 232. When thestructure is partly changed, it can be easily applied to the sub blockforming portion 230 and the activity portion 205.

A luminance level value Yk of one pixel of a sub block of fourpixels×four lines is input to a delay circuit 31 and one input terminalof an adding device 33. Data that is output from a selector 32 is inputto the other input terminal of the adding device 33. The adding device33 adds the luminance level value Yk and the data that has been outputfrom the selector 32. An output value of the adding device 33 istemporarily stored in a register 34 and then sent to the selector 32.The adding device 33, the selector 32, and the register 34 that are as aloop obtain the sum of 16 luminance level values Yk of one sub block offour pixels×four lines.

After the sum of the one sub block has been calculated, the selector 32outputs an initial value, 0. With the initial value, the adding device33 and the register 34 are initialized.

An output of the register 34 is supplied to a shift circuit 35. Theshift circuit 35 shifts the output of the register 34 by four bits so asto perform a calculation of 1/16. As a result, the average value P ofthe luminance level values of the sub block is obtained. The averagevalue P is input to a subtrahend input terminal of a subtracting device36. A luminance level value Yk delayed for a time period correspondingto the summing calculation of the sub block by the delay circuit 31 isinput to a minuend input terminal of the subtracting device 36.

The subtracting device 36 subtracts the average value P, which has beensupplied from the shift circuit 35, from the luminance level value Yk,which has been output from the delay circuit 31. An output of thesubtracting device 36 is supplied to an absolute value calculatingcircuit 37. The absolute value calculating circuit 37 obtains theabsolute value of the output of the subtracting device 36. The absolutevalue is referred to as absolute value of difference value, {abs(Yk−P)}. The absolute value of difference value, {abs (Yk−P)}, is inputto one input terminal of an adding device 40 through a register 38.

Data that is output from a selector 39 is input to the other inputterminal of the adding device 40. The adding device 40 adds the absolutevalue of difference value, {abs (Yk−P)}, and the data that has beenoutput from the selector 39. An output of the adding device 40 istemporarily stored in a register 41. The output of the adding device 40is input to the selector 39 and a shift circuit 42. The adding device40, the selector 39, and the register 41 as a loop obtain the sum of 16absolute values of difference values, {abs (Yk−P)}, of the sub block offour pixels×four lines.

After the sum for one sub block has been calculated, the selector 39outputs an initial value, 0. With the initial value, the adding device40 and the register 41 are initialized.

The sum of the absolute values of difference values, {abs (Yk−P)}, forone sub block, which have been output from the register 41, is suppliedto the shift circuit 42. The shift circuit 42 shifts the sum by fourbits so as to perform a calculation of 1/16. As a result, an averagedifference value var_sblk of the sub block is obtained. The averagedifference value var_sblk is supplied to a minimum value calculatingcircuit 43. The minimum value calculating circuit 43 selects the minimumaverage difference value var_sblk of the macro block to which a subblock that represents the supplied average difference value var_sblkbelongs. The selected average difference value var_sblk is output as anactivity act of the macro block.

The forgoing example shows that the process of the MPEG encoder 103 isimplemented by hardware. However, according to the present invention,the process of the MPEG encoder 103 is not limited to such an example.In other words, the process of the MPEG encoder 103 can be implementedby software. In this case, for example, a computer apparatus is providedwith analog and digital interfaces for a video signal. Softwareinstalled on the computer is executed with a CPU and a memory. In theforgoing digital VTR structure, the CPU and memory may be substitutedfor the MPEG encoder 103.

The software is recorded as program data on a recording medium such as aCD-ROM (Compact Disc-Read Only Memory). The recording medium, on whichthe software has been recorded, is loaded to the computer apparatus.With a predetermined operation, the software is installed onto thecomputer apparatus. As a result, the process of the software can beexecuted. Since the structure of the computer apparatus is well known,its description will be omitted in the following.

FIG. 19 is a flow chart showing an example of the process of the MPEGencoder 103 implemented by software. Since the process of the flow chartis the same as the process implemented by the forgoing hardware, theprocess of the flow chart will be briefly described in consideration ofthe process implemented by hardware. Steps S1 to S8 correspond to theprocess of the forgoing input field activity averaging process portion103A. Steps S11 to S22 correspond to the process of the forgoingpre-encoding process portion 103B. Steps S31 to S38 correspond to theprocess of the forgoing encoding process portion 103C.

At step S1, the first step, video data is captured. At step S2, the nextstep, each header of an MPEG EG is extracted in the vertical blankingregion and stored in a memory. In other than the vertical blankingregion, the captured video data is stored in the memory.

At step S3, video data of the first field is converted from raster scandata into block scan data. As a result, a macro block is extracted. Thisoperation is performed by controlling read addresses of the video datastored in the memory. At step S4, the extracted macro block issubdivided into sub blocks each of which is composed of four pixels×twolines.

At step S5, with the sub blocks into which the video data has beensubdivided, the activity act of the first field is calculated. In otherwords, as described above, the average value P of luminance level valuesYk of each sub block of four pixels×two lines is obtained. In addition,the absolute values of the difference values between the luminance levelvalues Yk and the average value P are obtained. With the absolute valuesof the difference values, the average difference value var_sblk of theluminance level values Yk of the sub block is obtained. The minimumaverage difference value var_sblk of one macro block is selected as theactivity act of the macro block.

The activity Activity (act) obtained at step S5 is cumulated at step S6and stored as the cumulated value sum to the memory. The process ofsteps S3 to S6 is repeated until it has been determined that the lastmacro block of the first field has been processed. In other words, thecumulated value sum is the sum of activities of macro blocks for onefield.

When it has been determined that the last macro block of one field hasbeen processed at step S7, the cumulated value stored in the memory isdivided by the number of macro blocks for one field at step S8. As aresult, the average value Activity (avr_act) of the field activities forthe field is obtained and stored in the memory.

When the average value Activity (avg_act) of the field activities hasbeen obtained, the flow advances to step S11. Like at step S3, at stepS11, the video data stored in the memory is converted from raster scandata into block scan data. As a result, a macro block is extracted.

At step S12, the field DCT encoding mode or the frame DCT encoding modeis selected. The selected result is stored as DCT mode type data dct_typin the memory. At step S13, corresponding to the selected result at stepS12, the macro block is subdivided into sub blocks each of which iscomposed of four pixels×four lines.

At step S14, with the sub blocks into which the video data has beensubdivided, the activity act of the macro block of the first and secondfields is obtained. In other words, as described above, the averagevalue P of the luminance values Yk of each sub block of four pixels×fourlines is obtained. In addition, the absolute values of the differencevalues between the luminance level values Yk and the average value P areobtained. With the absolute values of the difference values, the averagedifference value var_sblk of the luminance level values Yk of the subblock is obtained. The minimum average difference value var_sblk of themacro block is selected as the activity act of the macro block. With theactivity act and the average value Activity (avg_act) of the fieldactivities obtained at step S8 and stored in the memory, the normalizedactivity Activity (norm_act) is obtained. The obtained normalizedactivity Activity (norm_act) is stored in the memory.

At step S15, the macro block extracted from the video data at step S11is divided into DCT blocks each of which is composed of eightpixels×eight pixels. The DCT blocks are two-dimensionally transformedcorresponding to the two-dimensional DCT method. As a result, DCTcoefficients are obtained. At step S16, the DCT coefficients arequantized with a quantizer table (quantizer_table). Thereafter, the flowadvances to step S17.

The process of steps S17 to S21 is repeated with each quantizer scale(quantizer_scale) Qn. As a result, the processes corresponding to theforgoing Q_n portions 212, 212, . . . , the forgoing VLC portions 213,213, . . . , the forgoing cumulating portions Σ 214, 214, . . . , andthe forgoing cumulating portions Σ 215, 215, . . . . In other words, atstep S17, the DCT coefficients are quantized with quantizer scale Q=1.At step S18, the DCT coefficients quantized with reference to the VLCtable are encoded with a variable length code. At step S19, thegenerated code amount of the macro block with the variable length codeis calculated. At step S20, the generated code amount of each macroblock for one frame is cumulated. At step S21, it is determined whetheror not there is another quantizer scale Qn. When it has been determinedthat there is another quantizer scale Qn, the flow returns to step S17.At step S17, the process for the other quantizer scale Qn is performed.The generated code amounts for one frame corresponding to the individualquantizer scales Qn are stored in the memory.

When it has been determined that the cumulated value of the generatedcode amounts for the frame corresponding to all the quantizer scalevalues Qn has been obtained, the flow advances to step S22. At step S22,it is determined whether or not the last macro block (MB) of one framehas been processed. When the last macro block has not been processed,the flow returns to step S11. When the last macro block has beenprocessed and the generated code amount for one frame has beenestimated, the flow advances to step S31. At step S31, the real encodingprocess is performed.

Like at S11, at step S31, the video data stored in the memory isconverted from raster scan data into block scan data. As a result, amacro block is extracted. At step S32, corresponding to the DCT modetype data dct_typ stored in the memory at step S12, the DCT encodingmode is designated.

At step S33, the macro block extracted from the video data at step S31is divided into DCT blocks each of which is composed of eightpixels×eight lines. The DCT blocks are two-dimensionally transformedaccording to the two-dimensional DCT method. As a result, DCTcoefficients are obtained. The DCT coefficients are quantized with thequantizer table (quantizer_table) at step S34. Thereafter, the flowadvances to step S35.

At step S35, corresponding to the generated code amounts for one framecorresponding to the quantizer scales Qn, which have been estimated atsteps S11 to S22, the quantizer scales Qn used at step S36 aredesignated for each macro block so as to control the generated codeamount in the real encoding process.

Thereafter, the flow advances to step S36. With the quantizer scales Qndesignated at step S35, the DCT coefficients quantized with thequantizer table at step S34 are quantized. With reference to the VLCtable, the DCT coefficients quantized at step S36 are encoded with avariable length code at step S37. At step S38, it is determined whetheror not the last macro block of one frame has been processed. When it hasbeen determined that the last macro block of one frame has not beenprocessed, the flow returns to step S31. At step S31, the quantizingprocess and the variable length code encoding process are performed forthe next macro block. In contrast, when it has been determined that thelast macro block of one frame has been processed at step S38, theencoding process for one frame has been completed.

The forgoing example shows that the pre-encoding process of steps S11 toS22 is different from the encoding process of steps S31 to S38. However,it should be noted that the present invention is not limited to such anexample. For example, the generated code amounts estimated at steps S11to S22 are stored in the memory. Data that is obtained by the realencoding process is selected from the stored data. As a result, theprocess of steps S31 to S38 can be contained as a loop in the process ofsteps S11 to S22.

In the forgoing example, a macro block of 16 pixels×16 lines issubdivided into sub blocks each of which is composed of four pixels×fourlines. However, the present invention is not limited to such an example.A macro block may be subdivided into sub blocks each of which is smallerthan a sub block of eight pixels×eight lines. With the smaller subblocks, activities may be obtained. For example, an activity may beobtained in the unit of a sub block of (X/2^(n)) pixels×(Y/2^(m)) lineswhich is smaller than a DCT block of X pixels×Y lines.

In the forgoing example, as the picture data encoding system, the MPEG2system as an encoding system of which a picture is divided into blocksand each block is DCT-transformed is used. However, the presentinvention is not limited to such an example. As long as picture isencoded in the unit of one block, another encoding system can be used.

In the forgoing example of the present invention, a digital VTR has beendescribed. However, the present invention is not limited to such anexample. The present invention can be applied to a picture processingapparatus that compression-encodes picture data according to a blockencoding method, for example a picture transmitting apparatus thatcompression-encodes picture data and transmits the encoded picture data.

As described above, according to the present invention, since the sizeof a sub block with which an activity of a macro block is obtained isfour pixels×four pixels, which are smaller than the size of aconventional block of eight pixels×eight lines, the difference averagevalue of each sub block is more emphasized than that of each block ofeight pixels×eight lines. Thus, when the minimum value of differenceaverage values of sub blocks each of which is composed of fourpixels×four lines is selected as an activity of a macro block, theactivity of the macro block that contains even a small plain region canbe more suppressed.

This means that a quantizer value of a macro block that contains an edgeof a picture such as a character or a contour portion of a picture. As aresult, mosquito noise of a macro block that contains an edge portion ofa picture can be suppressed.

When mosquito noise takes place at a plain portion of a picture, ittends to be more remarkable. Thus, especially when the code amount issufficient and the compression rate is low, it is very effective tosuppress a quantizer value of a macro block that contains even a smallplain portion.

1. A picture processing apparatus comprising: transforming means forperforming a transforming process for picture data in the unit of atransforming block and outputting transformed coefficients as a resultof the transforming process; sub block forming means for forming a subblock from the transforming block; activity detecting means forcalculating an activity in the unit of a plurality of sub blockscontained in the transforming block of the picture data and outputtingan activity of a block composed of the plurality of transforming blockswith the activity of each of the sub blocks; and quantizing means fordeciding a quantizer scale of the block according to the activity outputfrom the activity detecting means and quantizing the transformedcoefficients output from the transforming means with the quantizerscale, wherein the activity detecting means comprises: means forobtaining an average value of levels of pixels for each sub block; meansfor obtaining an absolute value of a difference value between the levelof the pixel and the average value for each pixel; means for obtainingan average difference value of the absolute values of the differencevalues for each sub block; and means for selecting a minimum value fromthe average difference values of the block and using the selectedminimum value as the activity of the block.
 2. A picture processingcomprising: a transforming device configured to perform a transformingprocess for picture data in the unit of a transforming block andoutputting transformed coefficients as a result of the transformingprocess; a sub block forming device configured to form a sub block fromthe transforming block; an activity detecting device configured tocalculate an activity in the unit of a plurality of sub blocks containedin the transforming block of the picture data and outputting an activityof a block composed of the plurality of transforming blocks with theactivity of each of the sub block; and a quantizing device configured todecide a quantizer scale of the block according to the activity outputfrom the activity detecting device and quantizing the transformedcoefficients output from the transforming device with the quantizerscale, wherein the activity detecting device comprises: an averagingdevice configured to obtain an average value of levels of pixels foreach sub block; an absolute value device configured to obtain anabsolute value of a difference value between the level of the pixel andthe average value for each pixel; a calculating device configured toobtain an average difference value of the absolute values of thedifference values for each sub block; and a selecting device configuredto select a minimum value from the average difference values of theblock and using the selected minimum value as the activity of the block.